Cascaded Deep Convolutional Neural Network (DCNN) algorithm is the first model that uses Convolutional Neural Network (CNN) in facial landmark detection and the use of CNN improves the accuracy significantly. This strategy needs to perform regression processing to the data between the adjacent stages repeatedly, resulting in complex algorithm procedure. Therefore, a facial landmark detection algorithm based on Asymmetric Convolution-Squeeze Excitation-Next Residual Network (AC-SE-ResNeXt) was proposed with only single-stage regression to simplify the procedure and solve the non-real-time problem of data preprocessing between adjacent stages. In order to keep the accuracy, the Asymmetric Convolution (AC) module and the Squeeze-and-Excitation (SE) module were added to Next Residual Network (ResNeXt) block to construct the AC-SE-ResNeXt network model. At the same time, in order to fit faces in complex environments such as different illuminations, postures and expressions better, the AC-SE-ResNeXt network model was deepened to 101 layers. The trained model was tested on datasets BioID and LFPW respectively. The overall mean error rate of the model for the five-point facial landmark detection on BioID dataset was 1.99%, and the overall mean error rate of the model for the five-point facial landmark detection on LFPW dataset was 2.3%. Experimental results show that with the simplified algorithm procedure and end to end processing, the improved algorithm can keep the accuracy as cascaded DCNN algorithm, while has the robustness significantly increased.